add within group - #26781
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
…en/matrixone into 0807-add-within-group-by
|
|
There was a problem hiding this comment.
Requesting changes on exact head 15d9b16e832431baa3cdedc892813aab2ded3892 for two P1 blockers:
- Maximum-width DECIMAL inputs can still make
PERCENTILE_CONTreturn a discrete rounded value instead of the required continuous interpolation. - The contextual
WITHINlexer still breaks existing identifier use when the next token isGROUP.
I read the previous review, all review threads, and the PR conversation. The author replies explicitly address the previous findings, and the protocol gate, bounded spill path, basic identifier cases, and double-ORDER rejection are present; the two blockers below remain on the current head.
Review scope: full diff from merge-base 3c7392a0f86b23712da71f31d7a9198a2c8591a9, including parser → binder → plan/type inference → compile/remote protocol → aggregate merge/spill/flush/free and tests. All 26 GitHub checks are complete with no failing conclusion. Local targeted tests could not start in the isolated worktree because the required cgo/libmo.dylib artifact is absent; that is a test prerequisite failure, not test evidence. Both blockers below are directly provable from the current code paths.
|
… 0807-add-within-group-by
… 0807-add-within-group-by
iamlinjunhong
left a comment
There was a problem hiding this comment.
Reviewed exact head 902c821e4b5b63984df668ebb256f1e19c91275b against current origin/main 938a4dddd41298f53a067ae7ba6aef36b443564d (merge-base d01c85974f9540f8baa2e768ca86fea952941fe2). No P0/P1 blocker remains; one new P2 correctness issue is recorded inline, so this review is COMMENT rather than REQUEST_CHANGES.
I read the previous reviews, all review threads, and the PR conversation. The author reply after the prior review explicitly addresses both earlier P1s, and the current code/tests now reject PERCENTILE_CONT on DECIMAL(38,*), retain PERCENTILE_DISC support, and recognize only the full WITHIN GROUP ( phrase (including comments), preserving identifier use. Those prior blockers are resolved.
All 26 GitHub checks are complete with no failing conclusion, and git diff --check passes. Targeted local Go tests could not start in the isolated PR worktree because the required cgo/libmo.dylib artifact is absent; this is a prerequisite failure, not test evidence.
XuPeng-SH
left a comment
There was a problem hiding this comment.
Deep re-reviewed exact head 902c821e4b5b63984df668ebb256f1e19c91275b, including all existing reviews, inline comments, and author replies. The earlier protocol-version, bounded-spill, DECIMAL(38,*), contextual WITHIN, and double-ORDER findings are closed. One result-correctness blocker remains.
sortOrderedPercentileValues creates FLOAT32/FLOAT64 runs through mosort.SortByVectors, whose float ordering uses ordinary </> and therefore does not total-order NaN. The external merge heap uses cmp.Compare, which does total-order NaN. The merge consequently assumes a different ordering from the ordering used to create each run.
I reproduced this through the aggregate's public BulkFill/spill/Flush path with 20,001 FLOAT64 values and NaN at the first row of the second 10k batch: PERCENTILE_DISC(0) ascending returns 0 without spill but NaN after the 64-KiB spill boundary. The result therefore depends on input batching/memory pressure. Please use one explicit total comparator for both in-memory run sorting and merge (with defined ASC/DESC NaN semantics), and add spill/non-spill equivalence coverage for FLOAT32/FLOAT64 and CONT/DISC.
Focused parser, aggregate, group-spill, compile, and remote-protocol tests otherwise pass under controlled CGo; required CI is green and git diff --check passes.
aunjgr
left a comment
There was a problem hiding this comment.
Re-reviewed exact head 902c821e4b5b63984df668ebb256f1e19c91275b; all checks are terminal and successful. The earlier protocol, spill-bounding, parser compatibility, decimal-width, and double-ORDER findings are closed.
[P1] Use one total float order for both spill-run creation and k-way merging.
sortOrderedPercentileValues creates FLOAT32/FLOAT64 runs through mosort.SortByVectors, whose float comparators use ordinary < and > and do not order NaN. orderedPercentileRunHeap.Less merges those runs with cmp.Compare, which does order NaN. The merge therefore assumes an ordering that each input run does not satisfy. A supported NaN input can return a different percentile solely because the group crosses the spill threshold or lands on a different batch boundary.
Use the same explicit ASC/DESC total comparator in both paths, or reject NaN consistently, and add spill/non-spill equivalence coverage for FLOAT32/FLOAT64 and CONT/DISC.
… 0807-add-within-group-by
|
|
|
XuPeng-SH
left a comment
There was a problem hiding this comment.
Reviewed exact current head ab71d0495ed4069af1f1b6d60ebe5833290d0fc4.
The previous P1 NaN spill-ordering blocker is closed. In-memory sorting, spill-run generation, and merge-heap ordering now share compareOrderedPercentileValue, so FLOAT32/FLOAT64 NaNs have one total order across execution modes. The regression matrix covers FLOAT32/FLOAT64 × PERCENTILE_CONT/PERCENTILE_DISC × ASC/DESC and explicitly compares spill with non-spill results. The ordered-set remote protocol gates remain covered as well.
I re-ran the focused counterexamples plus the complete owning packages (aggexec, compile, plan, and the MySQL parser); all passed. Current-head CI is fully green. LGTM.
iamlinjunhong
left a comment
There was a problem hiding this comment.
Reviewed exact head ab71d0495ed4069af1f1b6d60ebe5833290d0fc4 against current origin/main 0f145bbc0a020e979e5bc2515de1a8a21ae4f222 (merge-base 494d77c8443b51694e0cbac7fc0a03b9a7f887eb). No P0-P3 finding remains.
I re-read my previous review, every review thread, and the PR conversation. After the prior P2 comment, the author explicitly replied with the corresponding fix: in-memory sorting, spill-run generation, and k-way merge now share compareOrderedPercentileValue; the regression matrix covers FLOAT32/FLOAT64 × PERCENTILE_CONT/PERCENTILE_DISC × ASC/DESC and spill/non-spill equivalence. The current implementation and tests close that issue.
I also rechecked the complete parser → binder/type inference → compile/remote protocol → aggregate fill/merge/spill/compaction/flush/free path, including H0 and single-hot-group behavior. The exact head is unchanged, git diff --check passes, and all 26 GitHub checks are terminal with no failing conclusion. LGTM.
Merge Queue Status
This pull request spent 17 minutes 12 seconds in the queue, with no time running CI. ReasonThe pull request can't be updated
HintYou should update or rebase your pull request manually. If you do, this pull request will automatically be requeued once the queue conditions match again. Requeued — the merge queue status continues in this comment ↓. |
Merge Queue Status
This pull request spent 1 hour 14 minutes 29 seconds in the queue, with no time running CI. Waiting for
All conditions
ReasonPull request #26781 has been dequeued Pull request from fork cannot be queued. This pull request comes from a fork, and Mergify needs the author's permission to update its branch.
HintYou should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it. Tick the box to put this pull request back in the merge queue (same as
|
What type of PR is this?
Which issue(s) this PR fixes:
issue #25144
#25110
What this PR does / why we need it:
新增 SQL WITHIN GROUP (ORDER BY ...) 语法及 AST、Binder、Explain 支持。
GROUP_CONCAT 支持两种排序写法:
新增 PERCENTILE_CONT 和 PERCENTILE_DISC 标量聚合,支持分组、ASC/DESC、NULL、数值类型及分布式合并。
百分位排序改为复用查询 ORDER BY 的 mosort.SortByVectors 实现。
增加聚合 spill、配置编码、资源释放和 DISTINCT group-concat spill 安全保护。
补充 parser、plan、executor、边界测试及 BVT 结果文件。
当前仍是 MVP:不支持百分位窗口函数、参数化 p、非数值排序列和 decimal256。